BP4D-Spontaneous: a high-resolution spontaneous 3D dynamic facial expression database

a r t i c l e i n f o Facial expression is central to human experience. Its efficiency and valid measurement are challenges that automated facial image analysis seeks to address. Most publically available databases are limited to 2D static images or video of posed facial behavior. Because posed and un-posed (aka " spontaneous ") facial expressions differ along several dimensions including complexity and timing, well-annotated video of un-posed facial behavior is needed. Moreover, because the face is a three-dimensional deformable object, 2D video may be insufficient, and therefore 3D video archives are required. We present a newly developed 3D video database of spontaneous facial expressions in a diverse group of young adults. Well-validated emotion inductions were used to elicit expressions of emotion and paralinguistic communication. Frame-level ground-truth for facial actions was obtained using the Facial Action Coding System. Facial features were tracked in both 2D and 3D domains. To the best of our knowledge , this new database is the first of its kind for the public. The work promotes the exploration of 3D spatiotem-poral features in subtle facial expression, better understanding of the relation between pose and motion dynamics in facial action units, and deeper understanding of naturally occurring facial action. Research on computer-based facial expression and affect analysis has intensified since the first FG conference in 1995. The resulting advances have made the emerging field of affective computing possible. The continued development of emotion-capable systems greatly depends on access to well-annotated, representative affective corpora [13]. A number of 2D facial expression databases have become available (e. Although some systems have been successful, performance degrades when handling expressions with low intensity appearance, large head rotation, subtle skin movement, and/or lighting changes with varying postures. Due to the limitations of describing facial surface deformation when 3D features are evaluated in 2D space, 2D images with a handful of features may not accurately reflect the authentic facial expressions (e.g., in-depth motion of 3D head pose, 3D wrinkles and skin extrusion in the areas of the cheek, forehead, glabella, nasolabial, and crow's feet). Another problematic issue is that facial action units (AUs) can occur in more than 7000 different complex combinations [17], causing bulges and various in-and out-of-image-plane movements of permanent facial features, negative emotions from view of the left hemiface, and mouth extrusions that are difficult to detect in a 2D plane. Three-dimensional dynamic surface analysis and tracking …